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^ (57) Abstract: system for caching data during peer-to-peer data transfer between computers in a network of computers, the system 
S including a database (12) for dynamically storing infonnation regarding the data available for transfer, proxy server (8) and cache 

manager (9), when combined, operable to transfer data selected from the available data to user (4) in response to a request f<»- the 
^ selected data fiom the user (4); and cache server (10) for storing a copy of data already transfetred by the system. The proxy server 

(8) and cache manager (9), in combination, are further operable to search the database (12) in response to a request for the data 
Q selected, to determine if the data selected is stored in the cache server (10), and to transfer the data selected from the cache server 

(10) to the user (4) if the data selected is stored therein, and, if the data selected is not available in the database (12), to tiasnsfer the 
^ data selected from a rranote user (5) in the networic having die data selected stored therein, to die cache server (10)and to the user (4). 
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Field of the Invention 

The present invention relates to a system for caching data during peer-to-peer data 
transfer in a server mediated peer-to-peer file-sharing system, particulariy, although not 
5 exclusively, using the Intemet, and for sharing audio files. 

Throughout the specification, unless the context requires othenvise, the word "comprise" 
or variations such as "comprises" or "comprising", will be understood to imply the inclusion 
of a stated integer or group of integers but not the exclusion of any other integer or group 
of integers. 

10 Background Art 

The following discussion of the background to the invention is intended to facilitate an 
understanding of the present invention. However, it should be appreciated that the 
discussion is not an acknowledgement or admission that any of the material referred to 
was published, known or part of the common general knowledge of the relevant person 
1 5 skilled in the art at the priority date of the application. 

The Intemet is a netwbric of computers that communicate via communication links such as 
a telephone network, and allow the exchange of information between the computers. The 
Worid Wide Web is a part of the Intemet, which allows networi^ed computers to send 
graphical images and data (including audio and video information) between the computers. 

20 Typically, computers - commonly referred to as servers - provide data (or Veb pages") in 
the form of text and graphical images that can be downloaded to remote terminals such as 
a personal computer ("PC"), where a user can view the web pages. These pages can be 
used to display infomnation on the display of the remote terminal, for example to give 
infonmation about a particular service provided by the host of a web site, or products and 

25 services provided by the host. The pages may include video and audio information and 
allow viewers of the web page to view the information. These web pages are viewed at 
the remote terminal using a so-called web browser. 
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Each web site (and individual web pages) is uniquely identifiable by means of a web 
address, or Unifonm Resource Locater ("URL"). Hyper Text Mark Up Language ("HTML") 
is a text and graphics formatting software that is commonly used to produce and view web 
pages, and the protocol used by web browsers and servers transferring HTML based files 
5 is the Hypertext Transfer Protocol ("HTTP"). In data transfer, when one application is in 
communication with another on a host computer, that application is specified in each data 
transmission by using its so-called port number. The well-known port number for HTTP 
transmission is port 80. for example. 

The Transmission Control Protocol - Internet Protocol ("TCP/IP") is the standard 
10 communications protocol for sending data over the IntemeL As is well known in the art, 
protocols that work together as a group are commonly referred to as a protocol stack, wrth 
different layers. The HTTP protocol operates at a higher layer than the TCP/IP protocol. 

Data is sent in packets (datagrams), which are routed via an Internet Protocol address ("IP 
address"). The required IP address is obtained by cross-referencing the specified URL. 

15 When a user wishes to access the Internet they usually connect to an Internet Service 
Provider ("ISP"). The ISP receives a request from the user to access a specific web site 
and connects the user's tenninal to the required server having the requested URL. The 
method of connection and subsequent communication is as has been described above. 

However, to reduce the amount of information that is transmitted over the Intemet, ISPs 
20 use a caching system where data that is frequently accessed is stored at the ISP. When 
that data is next requested by a user, it is provided from the local cache rather than 
connecting the user's tenminal to the required server. Existing caching systems use the 
known port number of HTTP transfers to capture requests for data and to redirect those 
requests to a local cache, if appropriate. 

25 The Intemet can also be used to ^cirrtate the sharing of files between users. One popular 
use is the sharing of audio files - typically those in the so-called MP3 fonfnat - using a 
server mediated peer-to-peer file sharing system. In this case, users access a Web site 
for details of other remote users connected to the web site who are willing to share files, 
the user then accesses another remote user's computer directly and downloads the 
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required files from that remote users cx)mputer. This is illustrated in Figure 1, and is 
described in more detail below. 

In the known file-sharing system 1. a user 4 accesses a file-sharing system meta-server 2 
using the Internet and the services of his ISP. Typically, the user 4 will be accessing the 
5 meta-server 2 using a terminal such as a personal computer equipped with a modem and 
the usual user interfaces as is well known. The meta-server 2 will retum the address of 
one or more file-sharing system servers 3 which the user 4 will use to access details of 
files available for download. Upon connection to the system server 3 - using for example, 
known login techniques - the user 4 will search or browse for files of interest and available 

10 for sharing by other remote users 5 of the system and cun-ently connected to it. This is 
done by sending either a SEARCH_REQUEST or BROWSE_REQUEST to the system 
server 3. In a SEARCH_REQUEST the user 4 will specify certain characteristics of a file 
that Is required, and the lists of all remote users 5 connected to the system servers 3 will 
be examined to see if a requested file, with these characteristics, is available to share. In 

15 a BROWSE.REQUEST the user will supply details of a particular known remote user, and 
all files that the remote user (if connected to the system server 3) is willing to share will be 
examined. The system server 3 then issues either a SEARCH^RESPONSE or 
BROWSE.RESPONSE back to the user 4 containing information on the files that are of 
interest to the user 4. Typically the results of the search or browse operation will be 

20 displayed on the user's remote tenminal. if the user wishes to download one of the files, 
he issues a DOWNLOAD_REQUEST to the system server 3. The system server 3 then 
issues a DOWNLOAD_ACK message that contains the address of the remote user 5 who 
has the file available for download. The user 4 - vvithout intervention from the system 
server 3 - makes a connection to the remote user 5 and the file is copied to the user 4. 

25 MP3 audio files are often large in size, and, because this type of file sharing can be 
extremely popular, it can use a significant proportion of an ISP's bandwidth. Because this 
file transfer is peer-to-peer, it is not earned over any specific port - the port to be used for 
a particular file transfer is negotiated between the two communicating peers on a transfer 
by transfer basis. This makes it difficult to cache. Further, the data is transient - it is only 

30 accessible while the remote user 5 that is sharing the file is connected to the Internet, 
further increasing the difficulty of caching the data. 
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Disclosure of the Invention 

According to a first aspect of the present invention, there is provided a system for caching 
data during peer-to-peer data transfer between computers in a network of computers, the 
system including: 

5 a first database for dynamically storing information regarding the data available for 

transfer from one or more computers in the network of coimputers; 

control means operable to transfer data selected from the available data to a first 
computer in the network in response to a request for the selected data from the first 
computer; and 

10 storage means for storing a copy of data already transferred by the system; 

wherein the control means is further operable to search the first database in response to a 
request for the selected data, to determine if the data is stored in the storage means, and 
to transfer the data from the storage means to the first computer if the data is stored 
therein, and, if the data is not available in the first database, to transfer the selected data 
15 from another computer in the network having the selected data stored therein, to the 
storage means for storage therein, and to the first computer. 

This has the advantage of allowing data that has already been transfenred by the system 
to be easily downloaded to additional users without having to connect to the original 
storage medium. Thus, files shared through a peer-to-peer file transfer system can be 
20 easily cached. 

Preferably, the control means comprises a first server and a second server, the first sen/er 
being coupled to a remote server for receiving information regarding the available data 
selected therefrom and for storing the infomnation in the first database, the second server 
including a second database having information to identify the data selected stored in the 
25 storage means and their storage locations therein and being coupled to the first sen/er for 
communication therewith. 
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More preferably, the second database identifies the data selected stored in the storage 
means by means of hashing methods or other unique identified encoding methods. 

Still more preferably, the second database identifies the data selected stored in the 
storage means by means of an MD5 hash value, 

5 Preferably, the second server may be operable, in response to a request from the first 
sender, to search the second database for the identifying data to thereby detemriine if the 
data selected is stored in the storage means, and its storage therein. 

Preferably, the second server may be operable, if the data selected is determined to be 
stored in the storage means, to send information on the storage location to the first server, 
10 the first server being further operable, in response to the received storage location 
infomiation, to send tills infonnation to the first computer. 

Preferably, the second server may be operable, if the selected data is determined not to 
be stored in the storage means, to send information on an available for storage location for 
the selected data to the first server, the first server being further operable, in response to 
15 the received storage location information, to send this information to the first computer. 

According to a second aspect of the invention there is a method of caching data during 
peer-to-peer data ti^nsfer including: 

selecting data to transfer; 

searching a first database of dynamically stored information regarding data 
20 available for transfer from one or more computers in a network of computers 

to detemnine if the selected data is stored in a storage means; 

if Uie search of One first database determines that the selected data is stored 
in the storage means, transferring the selected data from the storage means 
to a first computer in the network of computers; and 

25 if the search of the first database determines that the selected data is not 

stored in the storage means, transfem'ng the selected data from anotiier 
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computer in the network of computers having the selected data stored 
therein to the storage means and to the first computer. 



Preferably, the method includes the steps of 



receiving information regarding the data available for transfer from a remote 
5 server, and 

selecting data to transfer from the received information. 

According to a third aspect of the invention there is provided data transferred between 
computers in a network of computers, the data having been transferred by a control means 
in response to a request from a first computer in the network wherein the data has been 

10 selected from a set of dynamic data available for transfer stored in a first database and the 
control means has searched a storage means that stores a copy of data previously 
transferred by the control means to determine if the data is stored in the storage means 
and, if the control means determines that the data is stored in the storage means, to 
initiate the transfer of the data to the first computer from the storage means and, if the 

15 control means determines that the data is not stored in the storage means, to Initiate the 
transfer of the data to the first computer and the storage means from a second computer in 
the network having the selected data stored therein. 

Preferably, the control means determines whether the data is stored in the storage means 
by searching a second database containing infonnation that identifies the data stored in 
20 the first database and their storage locations. 

Brief Description of the Drawings 

One specific embodiment of the invention will now be described, by way of example only, 
with reference to the accompanying drawings, of which: 

Figure 1 is a schematic representation of a peer-to-peer file sharing system of the prior art; 
25 and 
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Figure 2 is a schematic representation of a system for caching data during peer-to-peer 
data transfer of the present invention. 

Best Mode(s) for Carrying Out the invention 

Figure 2 illustrates a system for caching data during peer-to-peer data transfer of the 
5 present invention. Those features that are known from the prior art, and discussed above 
in the preamble, will use the same reference numerals for clarity. 

In the embodiment, a user 4 is connected to the Internet via his ISP, in a known manner, 
and can access web sites by typing In the desired URL in his web browser. 

The ISP is provided with an ISP server 6 which acts as a gateway to provide connection to 
10 the Intemet for the user 4 (as is well known in the art). The ISP is also provided with a 
cache system, which communicates with the ISP server 6. The cache system comprises a 
proxy server 8, a cache director 7, a cache manager 9 and one or more cache servers 10 
- v^ose functions will be described in more detail below. 

When the user 4 wishes to download files from a remote user 5, he starts a file-sharing 
15 application that requests connection to a file sharing service using the TCP/IP protocol. 

In response to the connection request from the user 4. the ISP server 6 will redirect the 
user 4 to a cache director 7 for connection thereto. In response to this connection, the 
cache director 7 returns Oie address of the proxy server 8 via ISP server 6. The ISP 
server 6 is able to capture the initial connection request because it uses the TCP/IP 
20 protocol with a known port number. 

It should be appreciated that in other embodiments, the cache director 7 may be omitted. 
It is used in this embodiment to emulate the meta-server of existing peer-to-peer file 
sharing systems such as the Napster system operated by Napster Inc. of Redwood City, 
California. 

25 The user 4 then connects to the proxy server 8 via ISP sender 6. Upon receipt of a 
connection attempt from the user 4, the proxy server 8 connects to the meta-server 2 by a 
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direct TCP/IP connection and retrieves an IP address of a system server 3. Meta-server 2 
contains a large collection of valid system server addresses - the proxy server 8, retrieves 
one of these by querying meta-server 2 The proxy server 8 then makes a direct 
connection to the system server 3 using the address retrieved from its query of meta- 
5 server 2, 

The proxy server 8 acts as an intemiediary between the user 4 and the system server 3, 
and is operable to forward all messages it receives from the user 4 to the system server 3, 
and vice versa. These messages are forwarded verbatim until the system server 3 sends 
either a BROWSE_RESPONSE or SEARCH_RESPONSE message, in reply to a 
10 BROWSE_REQUEST or SEARCH_REQUEST message from the user 4 to the system 
server 3 to brov^e or search the files available for download. 

When a BROWSE^RESPONSE or SEARCH_RESPONSE message is received by the 
proxy server 8, the proxy server 8 is operable to store details of the files available for 
sharing in its memory. These details include the filename, the MD5 hash of the first 
15 300kB, the file size, encoding bitrate, length (time), the nickname (or nick) of the user 
sharing the file and the IP address of the user sharing the file. Some or ail of these details 
will also be displayed at the user's remote terminal in the usual manner. 

If there is a file that the user 4 wishes to download, then the user 4 issues a 
DOWNLOAD_REQUEST message that Is sent - via proxy server 8 - to system server 3. 
20 The system server 3 then responds with a DOWNLOAD_ACK message, which is sent to 
the proxy server 8. 

The proxy server 8, in response to the received DOWNLOAD^ACK message, connects to 
cache manager 9, which - In response - will return the address of one of the cache servers 
10 to the proxy server 8. 

25 The cache manager 9 includes a database 12. In the present embodiment, the database 
is an SQL database, but could be any mechanism for persistent storage. The cache 
manager 9 will accept one of four request types - namely, ASK=PROXY: ASK=CACHE, 
ASK=UPDATECACHED, AND ACTION=DELETECACHED. These will be discussed in 
more detail below. 
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The address returned by the cache manager 9 in response to a received message from 
the proxy server 8 will depend upon whether the file requested by the user 4 is already 
stored (or cached) locally in one of the cache servers 10, or whether it must be 
downloaded from a remote user 5 who has the file available for sharing. 

5 The proxy server 8 will send an ASK=PROXY request to the cache manager 9 to request 
a file wth a specific filename, from a specific remote user, with a specific MD5 hash value, 
in response to a received DOWNLOAD_ACK message from the system server 3. The 
cache manager 9 will search its database 12, firstly to determine if a file exists with the 
same MD5 hash value. If this is the case, the cache manager 9 will determine that it has 
10 found the requested file and will retum the address of the cache server 10 that contains 
that file. The MD5 hash is an algorithm used to provide a degree of certainty that two files 
are the same, and is well known to the person skilled in the art. 

If the cache manager 9 cannot detect a match for the MD5 hash value, then the cache 
manager 9 will attempt to find an approximate match. The algorithm for this is to, firstly, 

15 strip the path information from the filename, and attempt to match the filename exactly. If 
a match is found, then the cache manager 9 will then compare the size of the requested 
file with the size of the stored file and the encoding bit rate of the requested file with that of 
the stored file. If the encoding bit rates match, and the requested file size is no longer than 
that of the stored file (to prevent partially downloaded files being stuck in the cache), then 

20 a match will be recorded, and. again, the address of the cache server 10 containing the file 
will be returned. The algorithm is readily extendable to include matching on any of the 
stored details of the file. 

Should there be no exact or approximate match, then an assumption is made that the file 
is not stored in one of the cache servers 10 i.e. that it is not stored locally. In this case, the 
25 cache manager 9 selects a cache server 10 that has available storage capacity and the 
address of this cache server 10 will be returned - as it will be to this cache 10 that the 
requested file will be downloaded to. A record is then written into database 12 that 
contains details passed in the ASK request, including the address of the remote user 5 
from which the file should be downloaded. 
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In response to the received cache server 10 address, the proxy server 8 will rewrite the 
DOWNLOAD_ACK message with the address of the respective cache server 10 (rather 
than the address of the remote user 5, which will have been included in the 
DOWNLOAD_ACK message sent from the system server 3 to the proxy server 8). The 
5 proxy sender 8 will then send the revmtten DOWNLOAD_ACK message to the user 4. 

Upon receipt of the rewritten DOWNLOAD_ACK message, the user 4 will then attempt to 
directly connect to the particular cache server 10 (rather than the remote client 5), 
identified by the address inserted in the rewritten DOWNLOAD_ACK message from the 
proxy server 8. 

10 In response to a connection by the user 4, the cache server 10 issues a ASK=CACHE 
request to the cache manager 9 denoting a request to the cache manager 9 for the MD5 
hash value for the requested file. The cache server 10 will send the usemame of the 
remote user 5 and the full pathname for the requested file - as sent with the request from 
the user 4 to the cache server 10. in response to this request, the cache manager 9 will 

15 compare this data with that stored in the database 12, and retum the MD5 value. Using 
this MD5 hash value, the cache server 10 will search its own memory for a file identified by 
the retumed MD5 hash value. If the memory contains this file, then this is copied from the 
cache server 10 to the user 4 in the usual way. This second lookup is necessary so that 
the cache server can sen/e the correct file on request from the user - the protocol used in 

20 the Napster system refened to above specifies that download requests contain only the 
filename to be downloaded, while the cache server 10 only indexes on the MD5 sum. The 
cache manager 9, via the data stored in the database 12 is the only location where such a 
correlation can be made. 

If the cache server 10 does not find a copy of this file in its database from the retumed 
25 MD5 hash value, then it assumed that the file is not available locally and must be 
downloaded from the remote user 5. In this case, the cache sender 10 retrieves the IP 
address of the remote user 5 that is sharing the file from the information stored when the 
proxy server 8 received the DOWNLOAD_ACK message. The cache sen/er 10 then 
makes an outgoing connection to the remote user 5, and requests a copy of the file from 
30 the remote user 5 in a known manner. The downloaded file is copied to the user 4, and is 
also copied into the memory on the cache server 10 at the same time. 
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When the file has been copied to the memory, the cache server 10 will send an 
ACTION=UPDATECACHED message to the cache manager 9 Informing it of the file 
transfer, and including details of the newly stored file, so that it may be copied directly from 
the cache manager 10, should another request be made for that particular file. This flags 
5 that the file has been downloaded completely and therefore should not be cleaned up from 
the cache if a script were to be run cleaning the cache infonnation. Its explicit infomnation 
is to update the database state column in the database 12, for a particular file from 
•downloading" to "cached". 

If an error arose during download, then the cache manager 9 will also be infomied by an 
10 ACTION=DELETECACHED request from the cache server 10 to the cache manager 9. In 
this situation, it is undesirable for further requests to be directed to the partially 
downloaded file. The DELETECACHED clause directs the cache manager 9 to remove 
the database row in the database 12 referencing the partially downloaded file. If an error 
does occur, the user 4 will be advised accordingly and may retry If desired. 

15 All the messages and data transfer discussed above are carried out In accordance with the 
TCP/IP protocol. 

The cache manager 9 and cache servers 10 operate a least recently used policy to 
removes files from the cache as needed, as is known in the art 

It should be appreciated by the person skilled In the art that the scope of this invention is 
20 hot limited to the particular embodiment described above. In particular hashing methods 
other than the MD5 method may be used. Further, and in the altemative, unique identifier 
encoding methods other than hashing methods may be used. 
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Claims 

1 . A system for caching data during peer-to-peer data transfer between computers in a 
networic of computers, the system including: 

a first database for dynamically storing information regarding the data 
available for transfer from one or more computers in the network of 
computers; 

control means operable to transfer data selected from the available data to a 
first computer in the networic in response to a request for the selected data 
from the first computer; and 

storage means for storing a copy of data already transferred by the system; 

wherein the control means is further operable to search the first database in 
response to a request for the data selected, to determine if the data selected is 
stored in the storage means, and to transfer the data selected from the storage 
means to the first computer If the data selected is stored therein, and, if the data 
selected is not available in the first database, to transfer the data selected from 
another computer in the network having the data selected stored therein, to the 
storage means therein, and to the first computer. 

2. A system for caching data during peer-to-peer data transfer as claimed in claim 1, 
wherein the control means comprises a first server and a second server, the first 
server being coupled to a remote server for receiving information regarding the 
available data selected therefrom and for storing the information in the first 
database, the second server including a second database having information to 
identify the data selected stored in the storage means and their storage locations 
therein and being coupled to the first server for communication therewith. 

3. A system for caching data during peer-to-peer data transfer according to claim 2 
wherein the second database identifies the data selected stored in the storage 
means by means of hashing methods or other unique identified encoding methods. 
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4. A system for caching data during peer-to-peer data transfer according to claim 3 
wherein the second database identifies the data selected stored in the storage 
means by means of an MD5 hash value. 

5. A system for caching data during peer-to-peer data transfer according to any one of 
claims 2 to 4, wherein the second server may be operable, in response to a request 
from the first server, to search the second database for the identifying data to 
thereby detemiine if the data selected is stored in the storage means, and its 
storage therein. 

6. A system for caching data during peer-to-peer data transfer according to any one of 
claims 2 to 5. wherein the second server may be operable, if the data selected is 
determined to be stored in the storage means, to send information on the storage 
location to the first server, the first server being further operable, in response to the 
received storage location information, to send this information to the first computer. 

7. A system for caching data during peer-to-peer data transfer according to any one of 
claims 2 to 6. wherein the second server may be operable, if the selected data is 
determined not to be stored in the storage means, to send information on an 
available for storage location for the selected data to the first server, the first server 
being further operable, in response to the received storage location infonmation, to 
send this infonmation to the first computer. 

8. A method of caching data during peer-to-peer data transfer including: 

selecting data to transfer; 

searching a first database of dynamically stored information regarding data 
available for transfer from one or more computers in a networic of computers 
to determine if the selected data is stored in a storage means; 

if the search of the first database determines that the selected data is stored 
in the storage means, transferring the selected data from the storage means 
to a first computer in the networic of computers; and 
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if the search of the first database determines that the selected data is not 
stored in the storage means, transfemng the selected data from another 
computer in the network of computers having the selected data stored 
therein to the storage means and to the first computer. 

5 9. A method of caching data during peer-to-peer data transfer according to claim 8, 
further including the steps of 

receiving information regarding the data available for transfer from a remote 
server and 

selecting data to transfer from the received information. 

10 10. Data transfen-ed between computers in a network of computers, the data having 
been transferred by a control means in response to a request from a first computer 
in the network wherein the data has been selected from a set of dynamic data 
available for transfer stored in a first database and the control means has searched 
a storage means that stores a copy of data previously transfen'ed by the control 

15 means to determine if the data is storod in the storage means and, if the control 

means determines that the data is stored in the storage means, to initiate the 
transfer of the data to the first computer from the storage means and, if the control 
means detennines that the data is not stored in the storage means, to initiate the 
transfer of the data to the first computer and the storage means from a second 

20 computer in the network having the selected data stored therein. 

1 1 . Data according to claim 10 wherein the control means detennines whether the data 
is stored in the storage means by searching a second database containing 
information that identifies the data stored in the first database and their storage 
locations. 

25 12. A system for caching data during peer-to-peer data transfer substantially as 
described herein with reference to Figure 2. 
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A method of caching data during peerto-peer data transfer substantially 
described herein with reference to Figure 2. 
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